{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Pandas 字符串访问器基础\n",
        "\n",
        "本教程介绍Pandas中字符串访问器（str accessor）的基础知识，包括str访问器的介绍、使用方法、字符串类型转换以及NaN值处理。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 学习目标\n",
        "\n",
        "- 理解Pandas字符串访问器（str accessor）的概念和作用\n",
        "- 掌握str访问器的基本语法和使用方法\n",
        "- 了解字符串类型转换（object vs string dtype）\n",
        "- 掌握NaN值的处理方法\n",
        "- 理解字符串访问器的特点和优势"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 1. 导入库"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 2. 什么是字符串访问器（str accessor）\n",
        "\n",
        "Pandas的字符串访问器（str accessor）是一个强大的工具，它允许我们对Series中的字符串数据进行向量化操作。通过`.str`访问器，我们可以像操作单个字符串一样操作整个Series中的字符串，但操作是向量化的，性能更高。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 2.1 基本语法\n",
        "\n",
        "字符串访问器的基本语法是：`Series.str.方法名()`\n",
        "\n",
        "让我们先创建一个包含字符串数据的Series："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "原始数据:\n",
            "0        Python\n",
            "1        Pandas\n",
            "2         NumPy\n",
            "3    Matplotlib\n",
            "4       Seaborn\n",
            "dtype: object\n",
            "\n",
            "数据类型: object\n"
          ]
        }
      ],
      "source": [
        "# 创建一个包含字符串的Series\n",
        "data = pd.Series(['Python', 'Pandas', 'NumPy', 'Matplotlib', 'Seaborn'])\n",
        "print(\"原始数据:\")\n",
        "print(data)\n",
        "print(f\"\\n数据类型: {data.dtype}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 2.2 使用str访问器进行基本操作\n",
        "\n",
        "现在我们可以使用str访问器来操作这些字符串："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "转换为小写:\n",
            "0        python\n",
            "1        pandas\n",
            "2         numpy\n",
            "3    matplotlib\n",
            "4       seaborn\n",
            "dtype: object\n",
            "\n",
            "转换为大写:\n",
            "0        PYTHON\n",
            "1        PANDAS\n",
            "2         NUMPY\n",
            "3    MATPLOTLIB\n",
            "4       SEABORN\n",
            "dtype: object\n",
            "\n",
            "字符串长度:\n",
            "0     6\n",
            "1     6\n",
            "2     5\n",
            "3    10\n",
            "4     7\n",
            "dtype: int64\n"
          ]
        }
      ],
      "source": [
        "# 转换为小写\n",
        "print(\"转换为小写:\")\n",
        "print(data.str.lower())\n",
        "\n",
        "# 转换为大写\n",
        "print(\"\\n转换为大写:\")\n",
        "print(data.str.upper())\n",
        "\n",
        "# 获取字符串长度\n",
        "print(\"\\n字符串长度:\")\n",
        "print(data.str.len())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 3. 字符串类型转换（object vs string）\n",
        "\n",
        "在Pandas中，字符串数据默认使用`object` dtype存储。从Pandas 1.0开始，引入了专门的`string` dtype，它提供了更好的类型检查和性能。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "默认类型 (object):\n",
            "0     apple\n",
            "1    banana\n",
            "2    cherry\n",
            "dtype: object\n",
            "数据类型: object\n",
            "\n",
            "转换为string类型:\n",
            "0     apple\n",
            "1    banana\n",
            "2    cherry\n",
            "dtype: string\n",
            "数据类型: string\n"
          ]
        }
      ],
      "source": [
        "# 创建Series，默认是object类型\n",
        "series_object = pd.Series(['apple', 'banana', 'cherry'])\n",
        "print(\"默认类型 (object):\")\n",
        "print(series_object)\n",
        "print(f\"数据类型: {series_object.dtype}\")\n",
        "\n",
        "# 转换为string类型\n",
        "series_string = series_object.astype('string')\n",
        "print(\"\\n转换为string类型:\")\n",
        "print(series_string)\n",
        "print(f\"数据类型: {series_string.dtype}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 3.1 object vs string 的区别\n",
        "\n",
        "- **object dtype**: 传统方式，存储为Python对象，兼容性好\n",
        "- **string dtype**: 新的专用类型，提供更好的类型检查和性能，推荐使用\n",
        "\n",
        "两种类型都可以使用str访问器："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "object类型使用str访问器:\n",
            "0     APPLE\n",
            "1    BANANA\n",
            "2    CHERRY\n",
            "dtype: object\n",
            "\n",
            "string类型使用str访问器:\n",
            "0     APPLE\n",
            "1    BANANA\n",
            "2    CHERRY\n",
            "dtype: string\n"
          ]
        }
      ],
      "source": [
        "# object类型使用str访问器\n",
        "print(\"object类型使用str访问器:\")\n",
        "print(series_object.str.upper())\n",
        "\n",
        "# string类型使用str访问器\n",
        "print(\"\\nstring类型使用str访问器:\")\n",
        "print(series_string.str.upper())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 4. NaN值处理\n",
        "\n",
        "在处理字符串数据时，经常会遇到NaN（缺失值）。str访问器能够智能地处理这些缺失值。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "包含NaN的数据:\n",
            "0    Python\n",
            "1    Pandas\n",
            "2       NaN\n",
            "3     NumPy\n",
            "4      None\n",
            "dtype: object\n",
            "\n",
            "数据类型: object\n"
          ]
        }
      ],
      "source": [
        "# 创建包含NaN的Series\n",
        "data_with_nan = pd.Series(['Python', 'Pandas', np.nan, 'NumPy', None])\n",
        "print(\"包含NaN的数据:\")\n",
        "print(data_with_nan)\n",
        "print(f\"\\n数据类型: {data_with_nan.dtype}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 4.1 str访问器自动处理NaN\n",
        "\n",
        "str访问器会自动保留NaN值，不会抛出错误："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "转换为小写（自动保留NaN）:\n",
            "0    python\n",
            "1    pandas\n",
            "2       NaN\n",
            "3     numpy\n",
            "4      None\n",
            "dtype: object\n",
            "\n",
            "获取长度（NaN返回NaN）:\n",
            "0    6.0\n",
            "1    6.0\n",
            "2    NaN\n",
            "3    5.0\n",
            "4    NaN\n",
            "dtype: float64\n"
          ]
        }
      ],
      "source": [
        "# 对包含NaN的数据使用str访问器\n",
        "print(\"转换为小写（自动保留NaN）:\")\n",
        "print(data_with_nan.str.lower())\n",
        "\n",
        "print(\"\\n获取长度（NaN返回NaN）:\")\n",
        "print(data_with_nan.str.len())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 4.2 检查是否为NaN\n",
        "\n",
        "我们可以使用`isna()`或`notna()`来检查NaN值："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "检查是否为NaN:\n",
            "0    False\n",
            "1    False\n",
            "2     True\n",
            "3    False\n",
            "4     True\n",
            "dtype: bool\n",
            "\n",
            "检查是否不为NaN:\n",
            "0     True\n",
            "1     True\n",
            "2    False\n",
            "3     True\n",
            "4    False\n",
            "dtype: bool\n",
            "\n",
            "过滤掉NaN值:\n",
            "0    Python\n",
            "1    Pandas\n",
            "3     NumPy\n",
            "dtype: object\n"
          ]
        }
      ],
      "source": [
        "# 检查是否为NaN\n",
        "print(\"检查是否为NaN:\")\n",
        "print(data_with_nan.isna())\n",
        "\n",
        "# 检查是否不为NaN\n",
        "print(\"\\n检查是否不为NaN:\")\n",
        "print(data_with_nan.notna())\n",
        "\n",
        "# 过滤掉NaN值\n",
        "print(\"\\n过滤掉NaN值:\")\n",
        "print(data_with_nan[data_with_nan.notna()])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 5. 在DataFrame中使用str访问器\n",
        "\n",
        "str访问器同样可以在DataFrame的列上使用："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "原始DataFrame:\n",
            "      name              email      city\n",
            "0    Alice  alice@example.com  New York\n",
            "1      Bob       bob@test.com    London\n",
            "2  Charlie   charlie@demo.com     Tokyo\n",
            "3    David   david@sample.com     Paris\n",
            "\n",
            "各列数据类型:\n",
            "name     object\n",
            "email    object\n",
            "city     object\n",
            "dtype: object\n"
          ]
        }
      ],
      "source": [
        "# 创建包含字符串列的DataFrame\n",
        "df = pd.DataFrame({\n",
        "    'name': ['Alice', 'Bob', 'Charlie', 'David'],\n",
        "    'email': ['alice@example.com', 'bob@test.com', 'charlie@demo.com', 'david@sample.com'],\n",
        "    'city': ['New York', 'London', 'Tokyo', 'Paris']\n",
        "})\n",
        "\n",
        "print(\"原始DataFrame:\")\n",
        "print(df)\n",
        "print(f\"\\n各列数据类型:\\n{df.dtypes}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "将name列转换为大写:\n",
            "      name name_upper\n",
            "0    Alice      ALICE\n",
            "1      Bob        BOB\n",
            "2  Charlie    CHARLIE\n",
            "3    David      DAVID\n",
            "\n",
            "提取email域名:\n",
            "               email       domain\n",
            "0  alice@example.com  example.com\n",
            "1       bob@test.com     test.com\n",
            "2   charlie@demo.com     demo.com\n",
            "3   david@sample.com   sample.com\n"
          ]
        }
      ],
      "source": [
        "# 对name列使用str访问器\n",
        "print(\"将name列转换为大写:\")\n",
        "df['name_upper'] = df['name'].str.upper()\n",
        "print(df[['name', 'name_upper']])\n",
        "\n",
        "# 获取email的域名部分\n",
        "print(\"\\n提取email域名:\")\n",
        "df['domain'] = df['email'].str.split('@').str[1]\n",
        "print(df[['email', 'domain']])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 6. str访问器的特点和优势\n",
        "\n",
        "### 6.1 向量化操作\n",
        "\n",
        "str访问器提供向量化操作，比Python原生字符串方法循环处理要快得多："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "使用str访问器耗时: 0.0010秒\n",
            "使用列表推导式耗时: 0.0011秒\n",
            "\n",
            "性能提升: 1.09倍\n"
          ]
        }
      ],
      "source": [
        "# 创建大量数据用于性能对比\n",
        "large_data = pd.Series(['Python'] * 10000)\n",
        "\n",
        "# 使用str访问器（向量化）\n",
        "import time\n",
        "start = time.time()\n",
        "result1 = large_data.str.upper()\n",
        "time1 = time.time() - start\n",
        "print(f\"使用str访问器耗时: {time1:.4f}秒\")\n",
        "\n",
        "# 使用列表推导式（非向量化）\n",
        "start = time.time()\n",
        "result2 = pd.Series([s.upper() for s in large_data])\n",
        "time2 = time.time() - start\n",
        "print(f\"使用列表推导式耗时: {time2:.4f}秒\")\n",
        "print(f\"\\n性能提升: {time2/time1:.2f}倍\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 6.2 链式操作\n",
        "\n",
        "str访问器支持链式操作，可以连续调用多个方法：\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "原始数据:\n",
            "0      Python  \n",
            "1      PANDAS  \n",
            "2       numpy  \n",
            "dtype: object\n",
            "\n",
            "链式操作结果（去除空白 -> 转小写 -> 首字母大写）:\n",
            "0    Python\n",
            "1    Pandas\n",
            "2     Numpy\n",
            "dtype: object\n"
          ]
        }
      ],
      "source": [
        "# 链式操作示例\n",
        "text_data = pd.Series(['  Python  ', '  PANDAS  ', '  numpy  '])\n",
        "print(\"原始数据:\")\n",
        "print(text_data)\n",
        "\n",
        "# 链式操作：去除空白 -> 转小写 -> 首字母大写\n",
        "result = text_data.str.strip().str.lower().str.capitalize()\n",
        "print(\"\\n链式操作结果（去除空白 -> 转小写 -> 首字母大写）:\")\n",
        "print(result)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 7. 常见错误和注意事项\n",
        "\n",
        "### 7.1 非字符串类型使用str访问器\n",
        "\n",
        "如果Series中包含非字符串类型的数据，使用str访问器可能会出现问题：\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "混合类型数据:\n",
            "0    Python\n",
            "1       123\n",
            "2    Pandas\n",
            "3       456\n",
            "dtype: object\n",
            "数据类型: object\n",
            "\n",
            "转换为字符串后使用str访问器:\n",
            "0    PYTHON\n",
            "1       123\n",
            "2    PANDAS\n",
            "3       456\n",
            "dtype: object\n"
          ]
        }
      ],
      "source": [
        "# 包含数字的Series\n",
        "mixed_data = pd.Series(['Python', 123, 'Pandas', 456])\n",
        "print(\"混合类型数据:\")\n",
        "print(mixed_data)\n",
        "print(f\"数据类型: {mixed_data.dtype}\")\n",
        "\n",
        "# 需要先转换为字符串类型\n",
        "print(\"\\n转换为字符串后使用str访问器:\")\n",
        "print(mixed_data.astype(str).str.upper())\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 7.2 检查是否可以应用str访问器\n",
        "\n",
        "在使用str访问器之前，最好先检查数据类型：\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "数据类型: object\n",
            "可以使用str访问器\n",
            "0    PYTHON\n",
            "1    PANDAS\n",
            "2     NUMPY\n",
            "dtype: object\n"
          ]
        }
      ],
      "source": [
        "# 检查数据类型\n",
        "data = pd.Series(['Python', 'Pandas', 'NumPy'])\n",
        "print(f\"数据类型: {data.dtype}\")\n",
        "\n",
        "# 检查是否为object或string类型\n",
        "if data.dtype == 'object' or data.dtype == 'string':\n",
        "    print(\"可以使用str访问器\")\n",
        "    print(data.str.upper())\n",
        "else:\n",
        "    print(\"需要先转换为字符串类型\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 8. 实际应用案例\n",
        "\n",
        "### 8.1 数据清洗示例\n",
        "\n",
        "假设我们有一个包含用户信息的DataFrame，需要清洗数据：\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "原始数据:\n",
            "       username              email\n",
            "0    alice123    Alice@Example.com\n",
            "1     BOB_SMITH       bob@test.COM\n",
            "2     charlie     CHARLIE@demo.com\n",
            "3          None     eve@sample.com\n",
            "4         david               None\n",
            "\n",
            "清洗后的数据:\n",
            "       username username_clean              email        email_clean\n",
            "0    alice123         alice123  Alice@Example.com  alice@example.com\n",
            "1     BOB_SMITH      bob_smith       bob@test.COM       bob@test.com\n",
            "2     charlie          charlie   CHARLIE@demo.com   charlie@demo.com\n",
            "3          None           None     eve@sample.com     eve@sample.com\n",
            "4         david          david               None               None\n"
          ]
        }
      ],
      "source": [
        "# 创建包含需要清洗的数据\n",
        "users_df = pd.DataFrame({\n",
        "    'username': ['  alice123  ', 'BOB_SMITH', '  charlie  ', None, 'david'],\n",
        "    'email': ['Alice@Example.com', 'bob@test.COM', 'CHARLIE@demo.com', 'eve@sample.com', None]\n",
        "})\n",
        "\n",
        "print(\"原始数据:\")\n",
        "print(users_df)\n",
        "\n",
        "# 数据清洗：去除空白、统一大小写\n",
        "users_df['username_clean'] = users_df['username'].str.strip().str.lower()\n",
        "users_df['email_clean'] = users_df['email'].str.strip().str.lower()\n",
        "\n",
        "print(\"\\n清洗后的数据:\")\n",
        "print(users_df[['username', 'username_clean', 'email', 'email_clean']])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 9. 总结\n",
        "\n",
        "### 9.1 关键要点\n",
        "\n",
        "1. **str访问器语法**: 使用`Series.str.方法名()`来访问字符串方法\n",
        "2. **数据类型**: 字符串数据可以使用`object`或`string` dtype，两者都支持str访问器\n",
        "3. **NaN处理**: str访问器会自动处理NaN值，不会抛出错误\n",
        "4. **向量化操作**: str访问器提供向量化操作，性能优于循环处理\n",
        "5. **链式操作**: 支持链式调用多个字符串方法\n",
        "\n",
        "### 9.2 最佳实践\n",
        "\n",
        "- 优先使用`string` dtype而不是`object` dtype（Pandas 1.0+）\n",
        "- 在使用str访问器前，确保数据是字符串类型\n",
        "- 利用链式操作简化代码\n",
        "- 注意处理NaN值的情况\n",
        "- 对于大量数据，使用str访问器可以获得更好的性能\n",
        "\n",
        "### 9.3 下一步学习\n",
        "\n",
        "掌握了字符串访问器基础后，可以继续学习：\n",
        "- 字符串大小写转换（str.lower, str.upper等）\n",
        "- 字符串清理和空白处理（str.strip等）\n",
        "- 字符串查找和匹配（str.contains, str.find等）\n",
        "- 字符串替换和分割\n",
        "- 正则表达式操作\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "ml311",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}
